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ABSTRACT 

With the increase in institutional research activity 
and the number of reports resulting therefrom^ there is a growing 
need for better assessments of the impact of such reports* This paper 
proposes a framevork for analysis of institutional research report 
impact. The three major dimensions identified are: (1) time 
{short-term^ intermediate ^ and long-term) ; (2) action (accepted^ 
nonresponse^ and rejection) ; and (3) audience (primaiy or wirter/user 
the same^ secondary or written for someone else, and tertiary or 
written for someone else but passed on to and read by a third part) • 
A series of longitudinal case studies is proposed as c means of 
implementing the procedures described. A Research Impact Gram (RIG) 
that incorporates all three dimensions is described* Ti.e RIG draws on 
the sociogram techniques developed in the discipline of sociology* 
While the approach doe?^ uave iiwitations, it provides c simple 
technique for . determining the impact of institutional research 
reports. (DGC) 
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"*"°6oucaV.on'"' assessing impact of institutional studies • 



IV^?.r?SrT?S?tr6'."«S?r.'rs Lynn H. Willett (>wC> ^ 

i;V^".f.^c:.rs:'o'Xs^^ Elgin Conmnmity College 

coo.A..oNPos.r,osoapot.cv Elgin^ Illinois 

OsJ The establishment of institutional research offices in colleges and univer- 

^ sities has resulted in the dissemination of a large number of applied research 

^ studies. In most cases, these studies have been designed in response to local 

institutional needs and probletDS. Community needs assessment, graduate follow- 
^ up surveys, class size analyses, enrollment projections are among a few of the 

various types of reports written by institutional research staff. Typically, 
the internal dissemination of these reports is to selected but varied audiences. 

To assess the intact that these studies are having on the various audiences 
and institutions, a theoretical framework specifically designed for institutional 
research offices is needed. Report in?)act, defined as the varied behavioral and 
attitudinal response outcomes from audiences over a longitudinal time period, 
should be systematically evaluated as are other institutional outcomes. 

No theoretical model has been specifically developed to assess impact of the 
institutional researcher's report. Elaborate approaches, however, for assessing 
published research have been developed by researchers in assessing change (e.g., 
Guba, 1965) and in identi^ing communication processes (e.g., Lin, 1968). Trad- 
itional evaluation strategies, summarized by Worthen and Sanders 1973, focus on 
either institutional, program, course outcomes, or on processes. These strategies 
seem to offer little guidance to the institutional researcher in evaluating his 
report impact. 

In an extensive annotated bibliography on the research utilization process, 
Farr and Pingree, 1970, have identified over 200 relevant articles. Based on 
this literature, the authors formulated 15 propositions of research utilization.^ 
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^ (1) Presented at the American Educational Research Association Annual invention, 

O Washington, D.C., March 31 - April 3, 1975. 
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The en9>hasist however, of the propositions is on the coimnunication process 
rather than on the assessment of the report's impact. In a study more closely 
related to the institutional researcher's problems, Steinhoff (1968) used a case 
study methodology in a public school to assess federally funded project's imple- 
mentation; however, his focxis was on assessing communication strategies x»ather 
than report impact. 
IMPACT MODEL 

Drawing upon communication and decision-making literature, a three-dimensional 
model was developed to assist researchers in conceptualizing the research study 
impact process. The model (Figure 1) relates time, action and audience dimensions. 
The result of these relationships is defined as "institutional study impact." The 
components of the model are: 

Time ~ Van Dalen and Meyer (1966) cite a 25 year lag between dissemination 
of research studies reported in the literature and their eventual implementa- 
tion. Within an institution, no comparable time estimate has been made; how- 
ever, because of the nature of the document being disseminated, a multiple 
/ 

time-frame is advocated. Three time levels are identified: short (frm dis- 
semination until three months later), intermediate (from three months until a 
year later), and long-term (from one year until 5 years). 
Action ~ Havelock (1973) conceptualizes 6 stages in the process of an 
individual adopting a new idea. For purposes of a more simplified model, 
these six stages have been modified and merged into three action phases: 
"acceptance," where the audience has accepted the report and its findings and 
then takes an observable action; ^'non-response" revealed. through stalling or 
delaying tactics, raising questions designed to obscure the findings or simply 
ignoring the report; "rejection" evidenced by the audience's outright refusal 
to accept the report's findings or refusal to use the findings in any future 
actions. 
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Figure 1 
Model For Assessing Impact of 
Institutional Studies 
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Audience — Farr and Pingree (1970) identify four audiences for educational 
information - teachers, researchers, administrators and the public. For the 
proposed model, each of these four audiences at different times can comprise 
one of the three audience levels in the model. The various audiences are 
differentiated, not in terms of their traditional institutional roles, but 
in terms of their locus to the research requesting and disseminating process. 
The primary phase of the audience dimension represents the requesting individu- 
al or group which initiated the study. In many cases this would be the insti- 
tutional decision-maker to whom the director of institutional research reports. 
Secondary audiences are persons who routinely receive report copies, persons 
who immediately report tc the primary audience or persons co whom the primary 
audience reports. The tertiary or third audience level are people within the 
institution who are not immediately identifiable as report recipients or are 
not typical users of research report findings. 
METHODOLOGY 

A longitudinal cane study approach is the general research methodology used 
to implement the model. A variety of criterion measures and data collection in- 
struments are suggested for each time period to insure a reliable assessment of 
the research impact process. 

Short-Term ~ to obtain immediate attitudinal feedback from the primary group, 
a short checklist or rating scale could be used to inquire into the report's 
adequacy, understandability and relevance. Face-to-face interactions with 
this group would provide informal initial reactions and suggestions for report 
clarification. The intermediate audience can be identified from the report 
dissemination list. This group's attitudes could be solicited through the 
same rating scale used for the primary group, through a more general rating 
scale or through a semantic differential. Unobtrusive measures at this time 
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could be the appearance of administrative memos citing the report, number 
of people requesting the report and informal, unsolicited feedback. The 
tertiary audience can only be identified at this time. This can best be done 
by identifying persons who request the report, or by soliciting the primary 
and secondary groups to designate those persons with whom they have shared the 
report. 

Intermediate — more concrete forms of impact can be observed at this stage. 
Indirect observations of the various audiences will reveal whether the report 
findings are being acted upon. Typical questions would be: Have the results 
been referred to a committee? Have the results initiated or contributed to 
an external funding request? Each audience should be given another opportun- 
ity to give attitudinal feedback and to identify additional people who have 
become aware of the research report. 

Long-Term — in some cases a study may have had its final impact before reaching 
this time period. Most often, however, the lag between dissemination and im- 
plementation will make this phase the most fertile time for a study's impact. 
Indirect or anecdotal data gathering would be most frequently utilized at 
this level. Questions which would be helpful during this period of evalua- 
tion are: Have any observable changes been made in personnel, equipment, or 
procedures? Have any additional studies related to original study been re- 
quested? Are report copies still being requested? 
ANALYSIS 

In an effort to provide a comprehensive analysis mechanism, a Research Impact 
Gram (R.I.G.) has been adapted from the sociologists' sociogram (e.g., Gronlund, 
1959). R.I.G. is a comprehensive graphic analysis device which incorporates the 
time, audience and action dimensions. 
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To demonstrate R.I.G»'s potential, hjrpothetical data are depicted in Figure 2. 
The center point of the concentric circles represents the day the report is dis- 
seminated* Within each of the time circles, the audience types are represented 
usin^ different character designations • Straight line connections between the 
characters identify communication links within and between audience types. Aster- 
isks have been superimposed on the audience types when an observable action has 
occurred. In some cases, observable actions may result in a different time circle 
than when the audience first received the report. As will often happen in research 
dissemination, primary and secondary audiences will dominate the early time circles 
and tertiary audiences will be more abundant within the third time frame. 

R.I.G. provides an analysis of the total number of people who were exposed to 
the study by audience type as well as the observable actions by audience type. 
Ratio calculations can be determined for the number of persons exposed to the 
study and the number of actions. Also, R.I.G. can incorporate Halpert's (1966) 
suggestion that a research utilization index be calculated by dividing the number 
of units (audiences) applying the results of a particular piece of research by 
the total number of units capable of applying such results. 

The dynamic aspect of the research communication process can be depicted in 
R.I.G. As audience types are identified, characters are added and the chart 
expands accordingly. "Report Expansiveness Index" (R.E.I.) , a term adapted from 
sociometric group expansiveness analysis (Kerlinger, 196U),' can be calculated by 
comparing the number of tertiary audiences with their primary and secondary links. 
Studies with high R.E.I. 's would display large number of tertiary audience types 
in relation to the primary-secondary audiences. R.E.I, could also be considered 
a dissemination speed index by calculating the number of tertiary audiences in 
the short and intermediate circles. 
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Figure 2 
Research Impact Gram 
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In addition to R.I.G. other quantitative measures can focus on comparing 
attitudinal changes over time by audience type. Also, communication networks 
can be constructed which will assist in determining dissemination time estimates 
and communication links (Farr, 1967). Insights into the communication networks 
(both informal and formal) of an institution can provide the researcher with 
feedback to guide future disseminations. 

Qualitative analysis can focus on the written and verbal feedback pertaining 
to the report's content and format. Decision-making reactions can be categorized 
and labelled. These in time can be incorporated into R.I.G. 

Several data analysis limitations are inherent in this model. Measures be- 
come more unobtrusive as the audience goes from primary to tertiary. Data become 
less reliable as the time dimension lengchens, and action becomes more difficult 
to asress when the behavior is either non-responsive or rejecting. 
CONCLUSIONS 

Because of its simple conceptual framework the model has a mmiber of limit- 
ations. The impact process is a complex one involving a wide range of uncontroll- 
able variables which make causal analysis difficult. Organizational features 
impinge on the reporl" acceptance process, «3.g., the formal- informal decision- 
making processes, commitment to modem management techniques. Human dynamics and 
communication patterns which are difficult to study present a number of inter- 
vening variables. The nature of the report, how well it is written, the topic 
under consideration, and the extent of its initial dissemination could all influ- 
ence institutional impact. The nature of the audience in terms of its research 
sophistication and willingness to read and to use research results should also be 
considered in measuring report impact. 
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The model and its implementation would seem to imply a number of future 
studies. Does the impact of research findings vary directly in relation to the 
organizational level which requested it? Does the initial action by the primary 
audience affect the long-term impact of the study? Does the report form influ-- 
ence subsequent decision-making by audience level? Will the propostsd evaluetion 
process produce a "Hawthorne effect" which thereby increases the use of research 
results? 

The educational significance of the mod?l is that it provides a siniple con- 
ceptual frauework which can be implemented easily by institutional researchers 
in determining the impact of their research reports. In this era of accountability 
the institutional reseai^cher cannot avoid evaluation of one of the major outcomes 
of his job: the research report. 
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